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ABSTRACT 



The goal of forensics is to teach students the complexity 
and impact of communication on the human condition. The ballot is the key to 
the educational process within the competitive realm of forensics. The judge 
is both an adjudicator and a teacher within each round, and, therefore, 
ballots should provide students with comments indicating reasons for the 
judges' decisions; strengths and weaknesses of the student's abilities; and 
strengths and weaknesses of the student’s selection/material. Ballot comments 
in limited preparation events (extemporaneous speaking and impromptu 
speaking) provide a unique context for analysis. Data are from ballots 
written by judges during the 2000-2001 intercollegiate competitive season. 
Ballots were gathered after the completion of the season and are from a 
number of schools throughout the upper Midwest. The method used was based on 
Cary and Rodier's (1987) approach and Krippendorf ' s (1980) syntactical method 
of unit analysis. Evaluated were 110 ballots written for students competing 
in extemporaneous speaking. Eight primary categories were chosen, in 
hierarchical order, according to number of comments elicited: analysis; 
delivery; organization; sources; introduction; time; question; and 
miscellaneous. Also evaluated were impromptu speaking comments, and seven 
categories were generated: analysis; delivery; time; structure; introduction; 
quotation; and miscellaneous. Two major concerns were identified as 
illuminated by the data: the issue of "canned" speeches in extemporaneous 
speaking, and lack of an RFD (reason for decision) in impromptu speaking. An 
inordinate number of judges indicated they believed extemporaneous speeches 
were "canned, " while not one ballot analyzed in impromptu speaking provided 
an RFD. (Contains 34 references.) (NKA) 
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JUDGING THE JUDGES 

AN ANALYSIS OF BALLOTS IN IMPROMPTU AND EXTEMPORANEOUS SPEAKING 

Introduction 

Intercollegiate forensics is an educational enterprise. The goal of forensics is to teach students the 
complexity and impact of communication on the human condition. Students, whether engaged in 
prepared public address events, oral interpretation events, or limited preparation events, are expanding 
their understanding of the centrality of the communication process. The ballot is the key to the 
educational process within the competitive realm of forensics. Students should be able to understand 
how they performed in rounds, based on the comments judges write on ballots. Thejudge is both an 
adjudicator and a teacher within each round, and, therefore, ballots should provide students with 
comments indicating reasons for their decisions; strengths and weaknesses of the student’s abilities; and 
strengths and weaknesses of the student’s selection/material. Thejudge has the responsibility, therefore, 
to teach each competitor about critical issues within their events, their material and their performance 
through ballot comments. 

The overall importance of the ballot should not be underestimated. What a judge writes on a 
ballot has an impact on students. Rice and McGowan (1997) examined whether the comments judges 
write impact a students’ self-esteem. They determined both positive and negative comments had a 
measurable impact on students’ self-esteem. 

The importance both coaches and students place on ballots is evident in a study done by Peters 
(1998). Peters focused on how coaches and students determine the validity of judges’ comments. Peters’ 
research indicates three key factors used to evaluate ballot comments: (1) examining the nature of the 
comments; (2) the competitive reputation of a school/judge; and (3) comments from a former national 
champion or highly respected former competitor. 
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The ballot has become a significant component of intercollegiate forensics. The importance and 
centrality of the ballot should not be underestimated and is, therefore, worthy of extensive analytical 
evaluation. 



Review of Literature 

Research focusing on individual-event ballots, due to its obvious educational centrality to 
intercollegiate forensics, has received scrutiny from the discipline. Scholars in forensics, for example, 
have been quick to suggest ballot/judging criteria, uniform judging codes, and standards for ballot 
writing (Allen & Dennis, 1989; Bartanen, 1990, Hanson, 1989; Hanson, 1988; Hanson, 1987; Harris, 
1987; Jones, 1989; Kosloski, 1990; Lewis, 1984; Lewis, Keaveney & Leigh, 1984;Logue, 1988; ; Mills, 
1983; Mills, 1990; Murphy, 1988; Preston, 1983; Rice, 1992; Ross, 1984; Tucker, 1989; VerLinden, 
1987). 

Studies have explored the content of ballots focusing on original speaking events (Jensen, 1988; 
Pratt, 1987), communication analysis/rhetorical criticism (Dean & Benoit, 1984; Harris, 1987), and oral 
inteipretation events (Mills, 1991; Jensen, 1997a; Jensen, 1997b; Jensen, 1998). 

A particularly unique area of research is addressing the reason-for-decision (RED) on individual- 
event ballots. The RFD goes beyond more traditional ballot comments by indicating the precise reason a 
student earned a particular rank/rate in the round. The RFD, while a clear indicator of the competitive 
dimension of forensics also plays a critical teaching function. The student is able to learn from the RFD 
the reasons for a rank/rate, and take steps to address the issues raised. Scholars who have taken up the 
issue of RFDs include Burnett and Cronn-Mills, 1994; Hanson, 1997; Mills, 1990; Olson, 1992; Olson 
and Wells, 1988; Scott and Birkholdt, 1996. 
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Little explicit research has been conducted, however, in the area of ballot comments in limited 
preparation events. We contend limited preparation events deserve scrutiny. The two limited preparation 
events (extemporaneous speaking and impromptu speaking) provide a unique context for analysis. The 
limited preparation events are the only events in which the content of the speech is markedly different in 
every round of competition and, thus, the ephemeral nature of the event may influence the composition 
of comments judges may write. 



Method 

The data for our study are ballots written by judges during the 2000-2001 intercollegiate 
competitive season. The ballots are from a number of schools throughout the upper midwest. The ballots 
were gathered after the completion of the season. We sent a request to directors throughout the area to 
send all available ballots for inclusion in the study. The ballots are from a broad diversity of students and 
judges throughout the nation. 

Our method is based on the approach used by Carey and Rodier (1987) and Krippendorf s (1980) 
syntactical method of unit analysis. Each ballot was analyzed for its comments. A comment, according 
to Carey and Rodier (1987) is the “smallest unit possible.... if a comment was a restatement, it was still 
counted twice” (p. 5). 

Our study did not use pre-set categories, rather we allowed the categories to emerge from the 
data. Specifically, we used Berelson’s (1952) subject-matter orientation in which the categories are only 
limited by the imaginationof the researchers. 
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Results 

Extemporaneous Speaking 

We evaluated 1 10 ballots written for students competing in extemporaneous speaking. Our 
coding process resulted in 43 initial categories. Upon further reflection, we combined categories of a 
similar nature in order to better define the primary categories. The eight primary categories are listed 
below in hierarchical order, according to the number of comments elicited. 

Analysis 

Analysis comments question the information provided by the speaker along with any analytical 
questions the judge may have in regards to the speaker’s speech, or how the question was answered. 281 
total comments were written by judges about analysis issues. 

We identified six relevant patterns with the Analysis theme: 

(1) additional clarification needed (e.g., “what about Fox’s recent election? How will that effect 
economic growth?”); 130 comments 

(2) analytical compliments (e.g., “nice analysis of the economic restraints of communism in Russia”); 72 
comments 

(3) analysis effectively answers question; 61 comments 

(4) troubled thesis statement (e.g., “I don’t really understand what your answer to the question is”); 1 1 
comments 

(5) analytical repetition (e.g., “your second point is just like your first point”); 5 comments 

(6) audience analysis (e.g., “you should try to relate your question to your audience better”); two 



comments 
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Delivery 

Delivery comments highlighted the speakers’ overall verbal and nonverbal presentation within 
the rounds. We counted 265 total comments dealing with delivery-related issues. We identified eight 
patterns within the Delivery theme: 

(1) nice overall presentation; 101 comments 

(2) nonverbal (e.g., “make your gestures meaningful”; “put the pen down”); 60 comments 

(3) verbal flubs (e.g., “watch your enunciation, you’re swallowing words”); 32 comments 

(4) speed issues (e.g., “slow down, hombre”)\ 30 comments 

(5) word choice (e.g., “avoid using the word ‘like’”); 16 comments 

(6) energy (e.g., “nice energy”, “nice energetic style”); 1 1 comments 

(7) nice conversational tone; 10 comments 

(8) effective use of humor; 5 comments 

Organization 

Organization comments focused on the overall structure and framing of the speech. We 
identified 162 comments on organizational matters. A total of five patterns emerged dealing with 
organization and framing issues: 

(1) structure of speech (e.g., “strong analytical outline,” “speech seems disjointed”); 71 comments 

(2/3) transitions; 34 comments 

(2/3) summation and/or conclusion; 34 comments 

(4) preview statements; 14 comments 

(5) tag lines (e.g., “tag lines for your main points do not make sense”); 1 comment 
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Sources 

Sources comments addressed the overall use and validity of sources used or not used within the 
speech. We coded 142 comments relating to source issues. We noted six patterns in the Sources theme: 

(1) sources flowed by judge; 60 ballots had sources flowed 1 

(2) more sources needed; 39 comments 

(3) source credibility; 16 comments 

(4) diversity of sources; 12 comments 

(5) nice current sources; 8 comments 

(6) source allocation (e.g., “all of your sources are in your first point”); 7 comments 

Introduction 

Introductions comments focused on whether the introduction grabbed the audiences’ attention 
and linked to the question. A total of 1 10 comments were coded. We identified three patterns with the 
Introduction theme: 

(1) effectiveness of attention-getter; 68 comments 

(2) introduction relevance and link to question; 40 comments 

(3) requested a source citation in introduction; 2 comments 
Time 

Time comments deals with the competitors use and balance of time. We counted 47 comments 
addressing time factors, with three patterns: 

(1) time allocation (e.g., “work to balance time use in points one and two”); 33 comments 

(2) Length of speech (e.g., “speech was too short, you had more time to work with”); 7 comments 

(3) Overtime; 7 comments 
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Question 

Question deals with the competitor’s choice among the three questions provided. Nineteen 
comments were coded as dealing with the question. Three themes emerged from the data dealing with 

questions'. 

(1) significance of question; 7 comments 

(2) choice (e.g., “you should have chosen one of the other questions”); 6 comments 

(3) justification (e.g., “you need to justify why you chose the question”); 6 comments 

Miscellaneous 

Miscellaneous encompass the remainder of coded comments which do not fit within the themes 
established above. A total of 323 total comments are identified as miscellaneous. We address 9 patterns 
within the miscellaneous theme: 

(1) judge flowed speech; 102 

(2) judge timed speech; 95 2 

(3) ego-boosters (e.g., “great job, congrats on finals”); 51 comments 

(4) reason-for-decision provided; 25 comments 

(5) canned speech; 21 comments 

(6) competitor etiquette (e.g., “don’t fall asleep in rounds,” “wait until your called before going to 
the front of the room”); 15 comments 

(7) personal comments (e.g., “glad to see you again”); 5 comments 

(8) clothing (.e.g, “nice suit”) 3 comments 

(9) memorization (e.g., “lose the notecard”); 1 comment 
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Impromptu Speaking 

We evaluated 110 ballots written for students competing in impromptu speaking generating 1194 
total comments. Our coding process resulted in 45 initial categories. Upon further reflection, we 
combined categories of a similar nature in order to better define the primary categories. The seven 
primary categories are listed below in hierarchical order according to the number of comments elicited. 

Analysis 

Analysis comments address the overall analytical flow and problematic issues with the speaker’s 
explanation of the quotation. A total of 326 comments are coded as analytical. We identified six 
emergent patterns within the Analysis theme. 

(1) analytical concerns (.e.g., “you’re analysis does not address the second part of the 
quotation”); 180 comments 

(2) thesis (e.g., “what exactly is your thesis,” “get to your thesis sooner”); 57 comments 

(3) analytical compliments (e.g., “nice analysis here”); 50 comments 

(4) point balance (e.g., “your main points need to balance better analytically”); 19 comments 
(5/6) flow (.e.g, “your ideas don’t flow together very well”); 10 comments 

(5/6) subpoints (e.g., “your subpoints need to fit better under your main points”); 10 comments 

Delivery 

Delivery deals primarily with the overall verbal and nonverbal presentation of the speaker. A 
total of 221 comments were coded as addressing delivery-related issues. We coded 10 patterns within the 
Delivery theme. 
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(1) overall presentation; 85 comments 

(2) nonverbals (e.g., gestures, eye contact); 60 comments 

(3) word choice (e.g., “be sure verbs and nouns agree”); 24 comments 

(4) speech (e.g., “slow down”) 20 comments 

(5) humor (e.g., “add some humor,” “nice use of humor”); 10 comments 
(6/7/8) energy (e.g., “nice energetic style of delivery”); 6 comments 
(6/7/8) enunciation (e.g., “stop swallowing your words”); 6 comments 
(6/7/8) pronunciation (e.g., “watch overall pronunciation”) 6 comments 

(9) confidence; 3 comments 

(10) repetition (e.g., “stop repeating yourself’); 1 comment 

Time 

Time deals with the overall use of time and whether time was kept by the judge for the speaker. 
We coded 163 comments dealing with time factors. Five patterns emerged within the theme of Time. 

(1) overall time kept by judge; 65 

(2) prep time kept by judge; 55 

(3) time usage (“use your time better”) 28 comments 

(4) prep faster; 8 comments 

(5) overtime; 7 comments 

Structure 

Structure deals with the overall organization and framing of the speech with 128 comments 
identified. We noted six categories related to structure. 
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(1) conclusion; 33 comments 

(2) analytical framework (e.g., “I like the two-point analysis,” “a 3x1 is weaker than 2x2”; 30 
comments 

(3) preview statements; 28 comments 
(4/5) transitions; 16 comments 

(4/5) overall organization pattern; 16 comments 
(6) outline (e.g., “I like your outline”); 5 comments 

Introduction 

Introduction comments address how well the speaker begins their speech. 

(1) effectiveness of intro; 58 comments 

Quotation 

Quotation deals with the speaker’s choice of quotation and how they referred to it in the speech. 
A total of 31 comments in six patterns focused on the quotation. 

(1) references to quotation (e.g., “I like how you refer back to the quotation”); 12 comments 

(2) paraphrase (e.g., “you paraphrased the quotation well”) 10 comments 

(3) agree/disagree (e.g., “take a stand on the quotation”) 3 comments 
(4/5/6) choice (e.g., “always choose the shorter quotation”); 2 comments 
(4/5/6) quotation vs. quote (e.g., “it’s not a quote”); 2 comments 

(4/5/6) disagreement (e.g., “disagreeing with the quotation lets you stand out”); 2 comments 
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Miscellaneous 

Miscellaneous encompass the remainder of coded comments which do not fit within the themes 
established above. A total of 154 total comments are identified as miscellaneous. We address eight 
patterns within the miscellaneous theme. 

(1) speech flowed by judge; 80 comments 

(2) ego-boosters (e.g., “nice job”); 35 comments 

(3) etiquette (e.g., “please watch speakers after you”); 13 comments 

(4) personal (e.g., “nice to see you again”); 12 comments 

(5) audience rapport (e.g., “you have the audience in the palm of your hand”); 5 comments 

(6) clothing (e.g., “nice suit”); 2 comments 

(7/8) notecard (e.g., “lose the notecard”); 1 comment 

(7/8) credibility (e.g., “keep working on your credibility”); 1 comment 

Interpretation 

We identify two major concerns illuminated in the data: (1) issue of “canned” speeches in 
extemporaneous speaking and (2) lack of RFDs in impromptu speaking. First, an inordinate number of 
judges indicated they believed extemporaneous speeches were “canned.” We define a canned speech as a 
presentation which is prefabricated before the questions for the round were provided to the student. A 
common indicator of a canned speech was when the presentation was non-unique to the question posed. 
For example, a number of judges commented on how the speech was not specifically linked to the 
question. The lack of a clear link generated doubt in the mind of the judge that the speech was not 
actually prepared during the designated 1 /2-hour preparation period for the round. We are concerned 
about this issue on various levels: (1) are students pre-prepping speeches based on assumptions of what 
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the questions may be on any given weekend? (2) are coaches encouraging students to pre-prep 
extemporaneous speeches? (3) are coaches actually pre-prepping speeches themselves for their students 
to present? We encourage various national organizations to develop standards concerning the issue of 
canned speeches in extemporaneous speaking. Invitational tournament directors may decide to conduct 
an evidence check before the final round of extemporaneous speaking. We would encourage a variation 
of the rules of extemporaneous speaking employed by the National Forensic League (NFL). “Speakers 
shall prepare an original extemporaneous speech without references to previously prepared notes or 
speeches.” 3 We understand the enforcement of such a policy would be difficult for tournament officials, 
however the policy would, at minimum, set the ethical framework for the event. Current AFA-NIET 
tournament regulations do not specifically bar canned speeches. The 2001-2002 AFA-NIET invitation 
states, “[contestants] have 30 minutes to prepare a speech that is the original work of the student.” The 
NIET language does not bar students from pre-prepping speeches as long as the work is original to the 
contestant. 

Second, the study found out of the 110 ballots analyzed in impromptu speaking, not one ballot 
provided a reason for decision (RFD). Students, obviously, want to know how they did in the round, in 
direct comparison to the other competitors. The use of RFDs, we believe, is a critical component of the 
ballot and is essential in impromptu. Impromptu, by nature, is ephemeral. Since each impromptu speech 
and round is unique, it is essential for a student to know how their abilities compared to other students in 
that specific round. The use of RFDs is not new in forensics (see Burnett and Cronn-Mills, 1994; 
Hanson, 1997; Mills, 1990; Olson, 1992; Olson and Wells, 1988; Scott and Birkholdt, 1996), however, 
their application is clearly not present in contemporary practices. We encourage tournament directors to 
designate a specific portion of the ballot to RFDs. We recognize many tournaments already have a 
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designated RFD section. However, tournament directors need to request judges to complete the RFD 
sections of the ballot just as we request judges to properly fill in their master ballots. 
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